1. Field of the Invention
The present invention relates to a moving picture encoding system that can encode each of moving pictures applied thereto by using each of the input moving pictures and a predictive image which has been coded in advance of the encoding of each of the moving pictures. More particularly, it relates to a method of controlling the encoding of each of the moving pictures.
2. Description of the Prior Art
Referring now to FIG. 5, there is illustrated a flow diagram showing a prior art moving picture encoding system, which was adopted by TM5 of the international standard encoding system MPEG-2 and which is described in “The Foundations of Digital Image Compression”, Nikkei BP Publishing Center, pp. 189-196. Although the international standard encoding system MPEG-2 does not define a specific method of controlling the encoding of a motion video, that reference only discloses an encoding control method which was adopted by TM5 of MPEG-2 as a test model. The encoding control method, which was adopted by TM5 of MPEG-2, consists of three steps as shown in FIG. 5.
In step 1, in advance of encoding a picture of a certain type such as a picture (or I-picture) to be intra-coded using information only from that frame (or picture) or itself, a picture (or P-picture) to be encoded using a forward prediction, i.e. motion compensated prediction from a past frame, and a picture (or B-picture) coded using a forward or backward prediction, or both, i.e. motion compensated prediction from a past and/or future frame(s), global complexity measures defined by equations (1) to (3) are updated as:Xi=Si*Qi(ave)  (1)Xp=Sp*Qp(ave)  (2)Xb=Sb*Qb(ave)  (3)where Si, Sp and Sb are the number of bits generated by encoding this picture and Qi(ave), Qp(ave) and Qb(ave) are the average quantization parameter computed by averaging the actual quantiser step sizes mquant used during the encoding of all the macroblocks included in each frame, which are normalized into an integer in the range of 1 to 31. A moving picture that generates a larger amount of information when encoded provides larger global complexity measures Xi, Xp, and Xb. A moving picture that offers a higher degree of compression when encoded provides smaller global complexity measures Xi, Xp, and Xb. The respective global complexity measure is thus a normalized number relatively representing how much amount of information is required for encoding a picture of a certain type: an I-picture, a P-picture, or a B-picture, according to the type of that picture.
The global complexity measures have initial values Xi(init), Xp(init), and Xb(init) given by the following equations (4) to (6):Xi(init)=160*bit_rate/115  (4)Xp(init)=60*bit_rate/115  (5)Xb(init)=42*bit_rate/115  (6)where bit_rate is a bit rate (bps).
The respective target number of bits for the next picture in a Group of pictures or GOP (Ti, Tp, or Tb) can be computed by dividing the remaining number of bits allocated to the GOP by the number of pictures of the respective picture type (I, P, or B) to which the number of remaining I-, P-, and B-pictures is converted, as shown in equations (7) to (9). Assuming that all pictures not-yet encoded in the GOP are of the same type as that of the current picture currently being encoded, the respective target number of bits (Ti, Tp, and Tb) can serve as a guide for how many bits can be provided for one frame.Ti=R/(1+NpXp/XiKp+NbXb/XiKb)  (7)Tp=R/(Np+NbKpXb/KbXp)  (8)Tb=R/(Nb+NpKbXp/KpXb)  (9)where Kp and Kb are universal constants dependent on the quantization matrices. Typically, Kp=1.0 and Kb=1.4. Np and Nb are the number of P-pictures and B-pictures remaining in the current GOP in the encoding order. Furthermore, R is the remaining number of bits assigned to the current GOP. After encoding a picture, R can be given by any one of the following equations (10) to (12):R=R−Si  (10)R=R−Sp  (11)R=R−Sb  (12)where Si, Sp, or Sb is the number of bits generated in the I-, P-, or B-picture just encoded.
Before encoding the first picture in a GOP, the remaining number of bits R can be given by the following equations (13) and (14):R=G+R  (13)G=bit_rate*N/picture_rate  (14)where N is the number of pictures in the GOP.
Step 2 is the step in which the reference value of the quantiser step size for each macroblock is set by means of a virtual buffer. In performing step 2, the difference between the amount of information assigned to the current frame currently being encoded and the amount of information that has been generated actually in that frame is fed back in units of macroblocks. If the amount of information that has been generated actually in the picture just encoded is greater than the corresponding pre-programmed amount of information allocated to that picture, the quantiser step size is increased in order to reduce the amount of information which will be generated. Otherwise, the quantiser step size is decreased.
Before encoding the j-th macroblock, the fullness of an appropriate virtual buffer used for only the computation of the quantiser step size, which depends on the picture type (I, P, or B), is computed using a corresponding one of the following equations (15) to (17):di(j)=di(0)+B(j−1)−Ti*(j−1)/MB—cnt  (15)dp(j)=dp(0)+B(j−1)−Tp*(j−1)/MB—cnt  (16)db(j)=db(0)+B(j−1)−Tb*(j−1)/MB—cnt  (17)where di(0), dp(0), and db(0) are initial fullnesses of virtual buffers one for each picture type, B(j) is the number of bits generated by encoding all macroblocks in the picture up to and including the j-th macroblock, and MB_cnt is the number of macroblocks in the current picture. The final fullness of the virtual buffer is assumed to be the initial fullness (di(0), dp(0), or db(0)) for encoding the next picture of the same type.
The reference quantiser step size Qj of the j-th macroblock is computed using the following equations (18) and (19):Qj=d(j)*31/r  (18)r=2*bit_rate/picture_rate  (19)where d(j) denotes the fullness of the appropriate virtual buffer. The initial value for the virtual buffer fullness (di(0), dp(0), or db(0)) is given by a corresponding one of the following equations (20) to (22):di(0)=10*r/31  (20)dp(0)=Kp*di(0)  (21)db(0)=Kb*di(0)  (22)
In step 3, the reference quantiser step size computed in step 2 is modulated using a spatial activity measure for each macroblock. Following equations (23) to (25), the spatial activity measure for the j-th macroblock is computed from the four luminance sub-blocks in frame-based discrete cosine transform or DCT encoding mode, and the four luminance sub-blocks in field-based DCT encoding mode using the intra (i.e. original) pixel values:
                              abt          j                =                  1          +                                    min                                                sblk                  =                  1                                ,                8                                      ⁢                                          (                var_sblk                )                            ⁢                                                          ⁢              where                                                          (        23        )                                var_sblk        =                              1            64                    ⁢                                    ∑                              k                =                1                            64                        ⁢                                                  ⁢                                          (                                                      P                    k                                    -                  P_mean                                )                            2                                                          (        24        )                                P_mean        =                              1            64                    ⁢                                    ∑                              k                =                1                            64                        ⁢                                                  ⁢                          P              k                                                          (        25        )            and Pk are the pixel values in the original 8*8 block. Furthermore, the spatial activity measure for the j-th macroblock is normalized using the following equation (26):N—actj=(2*actj+avg—act)/(actj+2*avg—act)  (26)where avg_act is the average of actj for all macroblocks of the immediately preceding encoded picture. On the first picture, avg_act is 400.
The quantiser step size for the j-th macroblock mquantj is updated using the normalized spatial activity measure N_actj according to the following equation (27) as follows:mquantj=Qj*N_actj  (27)where Qj is the reference quantiser step size obtained in step 2. The final value of the quantiser step size thus obtained for the j-th macroblock, mquantj which is used to quantize the macroblock, is clipped to an integer that lies in the range of 1 to 31.
Japanese Patent Application Laying Open (KOKAI) No. 5-111012 discloses technology related to such a prior art moving picture encoding system. In this reference, it is described that the efficiency of encoding is improved without the slightest loss in picture quality when the ratio among the quantiser step sizes Qi, Qp, and Qb, i.e. Qi:Qp:Qb, used in the quantization process of quantizing I-pictures, P-pictures, and B-pictures is 1: αQ0: αQ, where αQ0 and αQ are constant numbers. It is further described in the above reference that both the distribution of the amount of generated codes among frames or fields and the expected value of the quantiser step size are determined on the basis of the quantiser step size used to encode a past frame or past fields, the amount of codes generated in the past frame or past fields, and the evaluated quantity or activity that is related to both the amount of codes to be generated in a future frame or future fields to be encoded and the amount of codes generated in the past frame or fields already encoded, and therefore the distribution of the amount of generated codes and the initial values of the initial value of the quantiser step size can be set so as to optimize the ratio among the quantiser step sizes used to quantize frames or fields using different predictive encoding methods. In addition, it is described that the ratio among the quantiser step sizes used for encoding an I-picture, a P1-picture, and a P2-picture approaches an optimum value as the amount of generated codes assigned to an I-picture is updated as previously mentioned, and the efficiency of encoding is therefore improved.
As previously mentioned, the prior art moving picture encoding system disclosed in Japanese Patent Application Laying Open (KOKAI) No. 5-111012 is so constructed as to set the distribution of the amount of codes among frames or fields and the initial value of the quantiser step size so that the ratio among the quantiser step sizes for the three different picture types is optimized, and update the distribution of the amount of codes to each I-picture in order to make the ratio among the quantiser step sizes used for quantizing I-pictures, P1-pictures, and P2-pictures approach an optimum value. Accordingly, the prior art moving picture encoding system differs from a moving picture encoding system according to the present invention in that the moving picture encoding system according to the present invention determines the ratio among the quantiser step sizes for the three different picture types according to one or more features of the sequence of moving pictures, as will be mentioned below.
The prior art moving picture encoding system that is so constructed as mentioned above controls the encoding operation according to the target number of bits for the respective picture type converted using the global complexity measures set for the three different picture types. As previously mentioned, a moving picture in which a larger amount of information is generated when encoded provides larger global complexity measures Xi, Xp, and Xb. In addition, a moving picture that offers a higher degree of compression when encoded provides smaller global complexity measures Xi, Xp, and Xb. Whether or not the quantiser step size directly related to the picture quality is controlled alone is not defined in step 1 performed when encoding each picture. In other words, the prime purpose of the encoding control of TM5 of the international standard encoding system MPEG-2 is to assign or allocate the target amount of codes to each of the three different picture types according to the respective global complexity measure X that is the product of the number of generated bits S and the respective quantiser step size Q. A problem with the prior art moving picture encoding system is thus that it does not independently control the quantiser step size that is used to quantize the macroblock and is related to the picture quality, and it is therefore difficult to control both the picture quality and the amount of generated codes.